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PREFACE 


This  briefing  describes  ways  in  which  the  Army  can  more  effectively  use 
the  National  Training  Center  (NTC)  for  focused  rotations  and  advanced 
warfighter  experiments.  The  work  reported  here  builds  on  a  decade  of 
Arroyo  Center  research  at  the  NTC,  which  has  been  sponsored  by  the 
Combined  Arms  Center,  Fort  Leavenworth,  and  the  Armor  Center,  Fort 
BCnox. 

This  project  was  sponsored  by  Major  General  Larry  G.  Lehowicz,  Deputy 
Chief  of  Staff  for  Combat  Developments,  TRADOC.  The  research  was 
conducted  in  the  Force  Development  and  Technology  Program  of  RAND's 
Arroyo  Center,  a  federally  funded  research  and  development  center 
sponsored  by  the  United  States  Army. 
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SUMMARY 


The  Army's  Combat  Training  Centers  (CTCs)  were  founded  with  a  dual 
purpose:  to  provide  the  most  realistic  training  exercises  possible  short  of 
actual  war  and  to  provide  "lessons  learned"  to  the  Army.  Although  the 
CTCs  have  excelled  in  the  first  objective,  they  have  been  less  successful  in 
generating  lessons  learned.  One  reason  for  this  is  the  absence  of  an 
analytic  data  base  capable  of  establishing  the  magnitude  of  a  range  of 
problems  and  helping  to  determine  how  they  may  be  solved. 

To  provide  such  a  data  base,  the  CTCs  have  begun  to  conduct  focused 
rotations  (FRs)  and  advanced  warfighter  experiments  (AWEs.)  The 
National  Training  Center  has  been  using  FRs  to  get  a  detailed  look  at 
battle  operating  systems  and  at  the  effects  that  specific  changes  in 
doctrine,  training,  organization,  leadership,  materials,  and  soldiers 
(DTOLMS)  have  on  BLUEFORCE  performance.  An  integral  part  of  Force 
XXI,  AWEs  have  a  broader  charter  to  examine  the  effects  of  multiple 
DTOLMS  changes. 

The  NTC  is  an  appropriate  site  for  warfighting  experiments.  It  provides 
an  intense  training  environment  in  which  units  fight  against  a  world-class 
opposing  force  (OPFOR).  The  NTC's  operations  group  is  able  to  perform 
independent  performance  assessments,  with  the  help  of  extensive 
instrumentation.  But  the  NTC  is  a  complex  laboratory,  and  tremendous 
care  must  be  taken  if  meaningful  experiments  are  to  be  performed  there. 

RAND's  many  years  of  conducting  research  at  the  NTC  have  granted  it 
substantial  expertise  in  conducting  effective  experiments  at  the  NTC,  both 
quantitative  and  qualitative.  RAND's  methodology  for  conducting 
research  at  the  NTC  can  be  used  to  plan  AWEs.  Primarily,  the  AWE  must 
have  objectives  that  can  be  adequately  demonstrated  at  the  NTC.  If  some 
of  its  measures  of  effectiveness  carmot  be  measured  or  are  compromised 
by  the  limitations  of  the  NTC's  training  environment,  then  those  objectives 
should  not  become  the  basis  for  an  FR  or  AWE.  If  the  research  objectives 
are  suitable,  a  train-up  plan  is  needed.  The  plan  should  produce  units  that 
are  well  trained  in  the  areas  to  be  tested.  In  addition,  the  test  plan  should 
make  maximum  use  of  archived  data,  and  the  data-collection  mechanisms 
should  be  established  and  verified  before  the  test. 

The  single-rotation  AWE  concept,  however,  even  at  its  best,  is  limited  in 
what  quantitative  results  it  can  provide.  RAND  has  found  that  at  least  12 
rotations'  worth  of  data  is  needed  to  provide  statistically  meaningful 


results.  The  AWEs,  however,  can  readily  provide  significant  qualitative 
results.  An  example  of  this  sort  of  experiment  would  be  to  provide  a  unit 
with  digital  reporting  equipment  and  determine  if  there  is  enough 
bandwidth  in  the  current  combat  radio  nets  at  peak  reporting  times 
during  the  battle.  While  the  AWE  would  not  establish  absolute 
bandwidth  requirements,  it  would  establish  the  need  to  relook  at  the 
combat  radio  net  requirements  for  a  "digital"  brigade. 

After  the  FR/ AWE  has  been  conducted,  analysis  is  needed  to  deal  with 
the  qualitative  results  obtained.  It  should  establish  what  changes 
observed  in  the  exercise  are  worth  pursuing;  simulations  can  help  refine 
the  expected  magnitude  of  these  changes  in  non-NTC  environments.  This 
briefing  supplies  an  illustration  of  the  methodology  applied  to  a  feasibility 
analysis  of  the  M1A2  tank. 

The  NTC's  training  mission  has  several  implications  for  the  planning  of 
FRs/AWEs.  It  is  a  training  environment,  and  although  it  approximates 
actual  combat,  it  does  not  replicate  it.  The  OPFOR  is  the  best-trained 
enemy  in  the  world,  and  its  advantages  must  be  considered  when 
planning  FRs/AWEs.  Moreover,  the  BLUEFORCE,  facing  this  opponent, 
loses  most  of  its  engagements.  This  promotes  highly  effective  training, 
since  units  learn  better  lessons  from  losing  than  from  winning,  but 
planners  must  account  for  the  reality  that  DTOLMS  improvements  gained 
in  an  FR  or  AWE  are  not  likely  to  result  in  a  BLUEFORCE  victory.  Other 
NTC  training  limitations  also  need  to  be  considered  in  the  design  of  an 
AWE.  BLUEFORCE  units  are  likely  to  be  organized  differently  than  they 
would  be  in  actual  wartime.  Training  at  the  NTC  focuses  only  on  certain 
missions.  A  battlefield  operating  system  that  receives  relatively  less 
emphasis  should  not  be  the  focus  of  testing.  Finally,  the  data-collection 
procedures  at  the  NTC  support  the  training  mission,  not  analysis. 

Planned  upgrades  to  NTC  instrumentation  are  expected  to  provide  more 
complete  and  more  detailed  data,  allowing  better  analysis  and  links  to 
simulation  systems. 

The  analytic  community  and  the  NTC  can  both  benefit  from  a  strong 
relationship.  Simulations  can  be  used  to  increase  the  effectiveness  of 
FRs/  AWEs;  the  intensity  of  the  NTC  training  supplies  a  rich  empirical 
data  base  for  the  analyst  to  draw  on  in  creating  simulations. 
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This  briefing  reports  the  results  of  a  study  to  determine  ways  in  which 
warfighting  experiments  conducted  at  the  NTC  could  be  improved. 
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Briefing  Outline 


Background 


»  Methodology  for  conducting  effective  AWEs  at  the 
NTC 


»  The  NTC  as  an  environment  for  experiments 

»  Strengthening  the  relationship  between  the  NTC  and 
the  analytic  community 


This  briefing  is  divided  into  four  sections.  The  first  section  discusses  the 
background  behind  the  current  Army  effort  to  conduct  AWEs  at  the 
NTC.  The  second  section  shows  how  effective  AWEs  can  be  conducted 
at  the  NTC.  The  third  section  describes  some  of  the  more  important 
AWE  issues  arising  from  the  NTC's  training  mission.  The  last  section 
discusses  how  the  analytic  community  and  the  CTC  can  each  benefit 
from  a  stronger  relationship. 
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The  Army's  Combat  Training  Centers 
Have  a  Dual  Purpose 

»  Conduct  realistic,  high-intensity  training  exercises 

»  Capture  lessons  learned 

-  Conduct  focused  rotations  (FRs)  and  advanced 
warfighter  experiments  (AWEs)  as  a  way  of 
providing  analytic  research  data  to  the  Army 


The  Army's  Combat  Training  Centers  (CTCs)  were  founded  with  a 
dual  purpose:  to  provide  the  most  realistic  training  exercises  possible 
short  of  actual  war  and  to  provide  "lessons  learned"  to  the  Army. 

The  CTCs  have  excelled  in  their  first  objective.  They  have  been  less 
successful  in  generating  lessons  learned.  Here,  an  important 
contributing  factor  is  the  absence  of  an  analytic  data  base  capable  of 
establishing  the  magnitude  of  a  range  of  problems  and  helping  to 
determine  how  they  may  be  solved. 

In  an  attempt  to  provide  such  a  data  base,  the  CTCs  have  begun  to 
conduct  focused  rotations  (FRs)  and,  more  recently,  advanced 
warfighter  experiments  (AWEs).  For  five  years  the  NTC  has  used  FRs 
as  a  way  of  getting  a  detailed  look  at  battle  operating  systems  and  at 
the  effects  that  specific  changes  in  doctrine,  training,  organization, 
leadership,  materials,  and  soldiers  (DTOLMS)  have  on  BLUEFORCE 
performance  at  the  NTC. 

An  integral  part  of  Force  XXI,  AWEs  have  a  broader  charter  to  examine 
effects  of  multiple  DTOLMS  changes. 
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Why  Conduct  These  Experiments  at  the  NTC?  ^ 


»  The  intensity  of  NTC  training  exercises  for  heavy 
brigades  is  as  close  to  war  as  is  possible  in  peacetime 

»  NTC  provides  independent  assessment  of  units' 
performance 

»  NTC  has  extensive  instrumentation 


For  several  reasons,  the  NTC  is  an  appropriate  site  for  warfighting 
experiments.  The  exercises  have  been  designed  to  represent  the 
conditions  of  war  as  closely  as  possible.  Units  fight  8  battles  in  14  days 
against  a  world-class  opposing  force  (OPFOR).  Many  Desert  Storm 
veterans  have  said  that  their  experience  in  ODS  was  good  training  for 
how  to  fight  at  the  NTC.  As  part  of  its  training  mission,  the  NTC  has 
developed  an  operations  group  that  consists,  in  part,  of  observer- 
controllers  (OCs)  who  independently  assess  the  performance  of  each 
NTC  unit,  from  platoon  to  brigade  level.  Moreover,  to  help  the  OCs 
perform  this  assessment,  the  NTC  has  developed  extensive 
instrumentation. 

However,  the  NTC  is  a  complex  laboratory,  and  tremendous  care  must 
be  taken  if  meaningful  experiments  are  to  be  carried  out  there. 
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RAND  has  conducted  both  qualitative  and  quantitative  research  at 
NTC  for  over  a  decade.  We  have  learned  how  to  select  topics  that  are 
important  to  the  Army  and  can  be  adequately  researched  in  this  "dirty 
laboratory."  RAND  research  in  each  topic  has  identified  systemic 
problems  within  BLUEFORCE  that  significantly  affect  its  combat 
performance  at  the  NTC.  We  have  successfully  designed  and 
conducted  experiments  at  the  NTC  to  determine  feasible  solutions  to 
these  problems.  The  results  of  this  research  have  led  to  changes  in 
Army  doctrine,  training,  organizations,  and  materials.  RAND  has 
published  reports  on  each  topic  as  well  as  an  overview  on  how  we 
conduct  research  at  the  NTC  (see  the  table  below  for  a  list).  This 
documented  briefing  is  based  on  this  successful  research. 

Topic _ RAND  Report  # 


Tactical  reconnaissance 

N  -2628-A 

Ground-to-ground  fratricide 

N-2438-A 

Artillery  targeting  accuracy 

N-2984-A 

TOW  missile  system  utilization  at  the  NTC 

N-3137-A 

Mortar  utilization  at  the  Combat  Training  Centers 

N-3358-A 

Battalion  and  below  C2  issues 

MR-496-A 

Quantifying  the  Battlefield:  RAND  Research 
at  the  National  Training  Center 

MR-105-A 
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This  section  shows  how  effective  AWEs  can  be  conducted  at  the  NTC. 
A  methodology  and  an  example  of  how  to  use  it  to  design  and  conduct 
an  AWE  are  presented. 


AWEs  Can  Provide,  Within  Certain  Limitations, 
Significant  Results  for  the  Army 


»  Not  every  system  or  concept  can  be  tested  at  NTC 

»  Measures  of  effectiveness  (MOEs)  can  be  compro¬ 
mised  by  limitations  in  the  NTC  training  exercises 

-  BLUEFORCE  combat  losses 

-  Artillery  accuracy 

-  Characteristics  of  MILES 

»  Statistically  significant  results  may  not  be  obtainable 

-  Baseline  data  usually  limited 

-  Single-rotation  AWE 

»  However,  important  qualitative  results  can  be 
obtained  by  using  all  data  sources 


The  NTC's  training  mission,  which  will  be  discussed  in  the  next  section, 
imposes  constraints  on  what  can  be  tested  at  the  NTC.  Understanding 
the  training  mission,  and  how  it  is  implemented,  is  required  to 
determine  what  can  be  tested  at  NTC.  If  the  training  mission  seriously 
affects  what  is  being  tested,  it  may  be  possible  that  the  AWE  rotation 
can  change  some  of  these  constraints  to  more  effectively  test  a  new 
concept  or  system. 

Care  must  also  be  taken  in  selecting  measures  of  performance.  An 
example  of  this  is  the  loss-exchange  ratio,  LER.  The  BLUEFORCE  fights 
until  no  further  training  can  be  gained.  Given  the  high  attrition  rate  for 
the  BLUEFORCE,  the  LER  will  be  highly  influenced  by  when  the 
exercise  was  stopped.  If  it  was  stopped  because  the  forward  companies 
were  attrited  and  the  reserve  company  was  too  far  back  to  get  in  the 
battle,  a  better  LER  will  be  obtained  than  if  all  the  companies  were 
involved  in  the  fight.  Another  factor  that  affects  the  LER  is  that 
artillery  lands  precisely  on  the  coordinates  given  in  the  call  for  fire 
request.  This  unrealistic  accuracy  often  favors  the  OPFOR,  whose 
knowledge  of  the  NTC  terrain  results  in  more  accurate  calls  for  fire. 

This  will  clearly  impact  the  LER.  Another  problem  exists  with  MILES 
(multiple  laser  engagement  system).  The  OPFOR  trains  with  MILES 
over  100  days  a  year,  as  opposed  to  the  BLUEFORCE,  which  typically 
receives  less  than  10  days  a  year  of  MILES  training.  The  OPFOR's 
superior  ability  to  use  MILES  against  the  BLUEFORCE  will  also 
adversely  affect  the  LER. 


The  concept  of  a  single-rotation  FR/ AWE  further  limits  the  NTC's 
analytic  usefulness.  RAND  has  found  that  data  from  at  least  twelve 
rotations  are  typically  needed  to  provide  statistically  significant  results. 
Unfortunately,  in  most  cases  no  baseline  data  exist,  and  it  is  difficult  to 
derive  statistically  meaningful  results  from  one  rotation.  Even  when 
there  are  baseline  data  for  a  specific  measure  of  effectiveness  (MOE), 
large  changes  in  the  MOE  during  the  rotation  are  needed  to  yield 
statistically  valid  results  at  the  .05  confidence  level.  The  AWE  must 
therefore  be  plarmed  to  take  advantage  of  the  archived  data  and  select 
MOEs  that  are  expected  to  change  significantly  during  the  AWE 
rotation.  Ideally,  the  critical  items  of  the  AWE  could  be  tested  over  a 
year's  worth  of  rotations,  which  would  allow  statistically  significant 
results  for  even  small  changes  in  the  MOE. 

This  is  not  to  suggest  that  FRs/  AWEs  are  limited  in  their  usefulness  to 
the  Army.  They  can  supply  important  insights  into  how  changes  in 
doctrine,  training,  organization,  leadership,  materials,  and  soldiers 
(DTOLMS)  can  affect  BLUEFORCE  performance.  In  many  cases  this 
may  be  the  only  way  to  gain  these  insights.  Quantitative  results  are 
also  possible,  but  a  structured  analytic  approach  is  required  to  design 
an  AWE  that  can  produce  meaningful  quantitative  results. 

It  is  also  important  to  note  that  the  insights  are  qualitative.  Further 
analysis  is  required  if  the  results  are  to  be  quantified.  AWEs  need  to  be 
designed  with  realistic  goals,  given  the  difficulty  of  obtaining 
quantitative  results. 
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The  Training  Mission  Introduces  Particular 

Limitations  for  Focused  Rotations  and  AWEs 

»  Training  missions  stress  only  certain  parts  of  the 
brigade 

»  Test  results  for  training  and  equipment  are 
confounded 

»  NTC  rules  of  engagement  can  limit  the  effectiveness 
of  both  the  new  equipment  and  the  training 


The  NTC  is  not  a  testing  and  evaluation  center.  Although  the  NTC 
operations  group  is  extremely  helpful  in  conducting  FR/ AWEs,  they 
will  not  compromise  their  training  function. 

In  the  next  section  I  show  that  BLUEFORCE  units  are  deployed  to  NTC 
with  an  organizational  structure  different  from  the  one  that  would  be 
used  in  a  real  war  and  that  this  presents  a  particular  problem  at  brigade 
level.  The  limited  set  of  missions  performed  by  an  NTC  brigade 
stresses  some  battlefield  operating  systems  (BOS)  more  than  others. 

This  limits  not  only  what  can  be  tested  but  also  to  what  level  additional 
objectives  can  be  tested. 

When  new  equipment  is  being  tested,  the  situation  is  even  more 
complex  because  at  NTC  it  is  difficult  to  separate  training  problems 
from  equipment  problems.  That  is,  the  equipment  may  work  as 
expected  but  the  battle  may  still  be  lost  because  the  units  could  not  use 
it  to  the  level  needed  to  influence  the  battle. 

NTC  rules  of  engagement  can  also  limit  the  effectiveness  of  a  new 
concept.  For  example,  increasing  the  accuracy  of  artillery-delivered 
mines  (FASCAM)  would  have  little  impact  on  the  battle,  since  the  rules 
of  engagement  require  that  no  obstacle  prevent  the  direct  fire  battle.  In 
practical  terms,  usually  only  the  first  vehicle  in  each  column  is  played 
as  being  attrited  by  the  FASCAM  in  the  NTC  exercise. 

The  above  are  just  some  of  the  factors  designers  must  take  into  account 
when  FRs  or  AWEs  are  to  be  conducted  at  the  NTC. 
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RAND  has  developed  over  the  last  decade  a  methodology  for 
conducting  research  at  the  NTC.  The  chart  shows  how  this 
methodology  can  be  used  to  plan  AWEs. 

If  the  objectives  of  an  exercise  cannot  be  adequately  demonstrated,  or  if 
their  MOEs  cannot  be  measured  or  are  compromised  by  limitations  of 
the  NTC,  that  exercise  should  not,  without  modification,  become  the 
basis  for  an  FR  or  AWE  at  the  NTC.  Even  if  the  objectives  are  feasible, 
they  should  result  in  substantially  improved  BLUEFORCE 
performance,  given  the  expense  of  conducting  FRs/AWEs. 

After  the  objectives  are  established,  a  train-up  plan  is  needed.  The  plan 
should  produce  units  that  are  well  trained  in  the  areas  to  be  tested. 

This  will  help  separate  training  issues  from  equipment  issues. 
Wherever  possible,  the  test  plan  should  make  maximum  use  of 
archived  data,  and  the  data-collection  mechanisms  should  be 
established  and  verified  prior  to  the  rotation. 

After  the  FR/ AWE  has  been  conducted,  a  substantial  analytic  effort 
will  be  required  to  deal  with  the  qualitative  results  obtained.  The 
analysis  should  establish  what  changes  observed  in  the  exercise  are 
worth  pursuing.  Simulations  can  help  refine  the  expected  magnitude 
of  these  changes. 
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Feasibility  Analysis  for  the  Ml A2  Tank 

M1A2  has  CITY,  POSNAV,  IVIS,  and  an  improved 
gunner's  sight 

»  Can  be  demonstrated  at  the  NTC? 

-  64  MlA2s  built 

-  M1A2  crew  training  program  in  place 

-  Field  exercises  limited 

-  Draft  of  Tactics,  Techniques,  and  Procedures 
manual  available 


The  Ml  A2  tank  can  be  used  to  illustrate  the  methodology.  A 
significantly  improved  version  of  the  Ml  Al,  the  Ml  A2  is  equipped 
with  a  commander's  independent  thermal  viewer  (CITY);  POSNAV,  an 
inertial  navigation  aid;  the  intervehicle  information  system  (IVIS),  a 
computer-based  reporting  system;  and  an  improved  two-axis  stabilized 
gunners'  sight.  The  Ml  A2  has  been  used  in  three  FRs  at  the  NTC.  The 
feasibility  analysis  presented  here  shows  how  the  M1A2  will  impact 
only  a  small  part  of  the  NTC  training  exercises. 

Sixty-four  Ml  A2s  have  been  built.  An  Ml  A2  training  program  has 
been  in  place  for  over  a  year,  and  more  than  100  tank  crews  have  been 
qualified  on  the  M1A2.  However,  qualifying  a  crew  does  not  ensure 
that  it  will  perform  well  in  the  NTC  environment.  The  small  number  of 
available  Ml  A2  tanks  has  limited  the  amount  of  time  that  units  have 
spent  in  field  exercises  with  the  tank.  Success  at  NTC  correlates  with 
the  number  of  field  exercises  performed  by  the  unit  at  home  station. 
Success  at  NTC  also  requires  good  tactics.  The  Tactics,  Techniques,  and 
Procedures  manual  for  the  Ml  A2  was  recently  released  in  draft  form, 
but  it  is  new  and  unproven,  and  it  is  not  clear  how  effective  it  will  be. 

Thus,  although  it  is  technically  possible  to  demonstrate  an  Ml  A2  TF  at 
the  NTC,  there  are  likely  to  be  some  training  and  tactics  issues  that  will 
reduce  the  tank's  effectiveness  in  a  focused  rotation. 
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Feasibility  Analysis  for  the  M1A2  Tank  I 

(cant'd)  I 


Key  M1A2  Improvements 


Measure  of  effectiveness 

Simnet 

demonstrated 

improvement? 

Measurable  at 
NTC? 

Credible 
measurement 
at  NTC? 

Mission  time 

20% 

Yes 

No 

Reporting  accuracy 

70% 

No 

No 

Target  acquisition 
range 

45% 

20%  of  time 

Yes 

Loss-exchange  ratio 

60% 

At  end  of 
mission 

No 

The  mounted  warfare  testbed  at  Fort  Knox  has  used  Simnet  to  show 
several  areas  where  the  M1A2  offers  major  improvements  over  the 
MlAl,  but  the  MOEs  are  not  all  measured  at  the  NTC.  Mission  time 
and  loss-exchange  ratio  are  measured  after  the  mission  has  exhausted 
all  training  opportunities.  They  are  therefore  inaccurate,  since  NTC 
battles  are  run  far  longer  than  real  battles  would  last. 

Reporting  accuracy  is  subjectively  rated  for  each  battle  by  the  OCs. 
Quantified  data  on  the  accuracy  of  reports  on  enemy  and  friendly 
positions  are  not  taken  at  NTC.  Target-acquisition  range  is  measured 
only  when  the  tank  kills  a  target,  and  then  only  if  the  Star  Wars 
computer  determines  a  pairing,  which  occurs  for  approximately  20 
percent  of  the  kills. 
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At  the  NTC,  the  terrain  itself  imposes  constraints  on  tanks.  With  a 
choice  of  wide-open  desert  or  mountain  passes,  the  OPFOR  tends  to 
force  the  battle  to  occur  in  the  passes,  where  target  ranges  are  less  than 
two  kilometers.  At  these  ranges,  the  optical  sights  for  both  the  OPFOR 
and  the  BLUEFORCE  are  roughly  comparable. 

Live-fire  safety  requires  that  tanks  fire  only  in  their  assigned  sectors. 
Because  the  operations  group  felt  that  commanders  using  CITV  could 
inadvertently  call  for  fire  out  of  sector,  the  CITV  system  was  "slaved" 
to  the  gunner's  sight,  thereby  limiting  the  target  acquisition  time  to  that 
of  an  MlAl.  Moreover,  at  the  NTC,  force-on-force  battles  are  rarely 
conducted  at  night,  so  the  superior  thermal  sights  on  BLUEFORCE 
vehicles  do  not  contribute  as  expected. 

Overall,  NTC  limitations  clearly  affect  the  potential  improvements  the 
Ml  A2  could  demonstrate  in  a  real  war.  Since  many  of  these 
improvements  are  in  the  range  of  20  to  40  percent,  it  is  possible  that  the 
Ml  A2  might  not  exhibit  any  improvements  over  the  Ml  A1  during  NTC 
battles.  The  only  part  of  the  training  exercises  that  is  not  subject  to  the 
constraints  discussed  above  is  the  counter-reconnaissance  fight,  which 
occurs  at  night.  Because  the  OPFOR  does  not  know  where  the 
BLUEFORCE  is,  its  ability  to  exploit  terrain  is  limited. 

In  this  case,  the  methodology  indicates  that  the  M1A2  focused  rotation 
should  concentrate  on  the  counter-reconnaissance  part  of  the  training 
exercise. 
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Briefing  Outline 

»  Background 

»  Methodology  for  conducting  effective  AWEs  at 
the  NTC 

»  The  NTC  as  an  environment  for  experiments 


»  Strengthening  the  relationship  between  the  NTC 
and  the  analytic  community 


The  methodology  presented  in  the  previous  section  highlights  the 
difficulties  in  conducting  experiments  at  the  NTC.  The  difficulties  are 
the  result  of  NTC's  primary  goal  of  providing  world-class  training  to 
the  U.S.  Army.  In  accomplishing  this  goal  the  NTC  has  limitations  that 
can  adversely  impact  any  AWE's  objectives.  This  section  presents  some 
of  the  more  significant  limitations.  In  many  cases  the  AWE  can  be 
modified  to  overcome  these  limitations,  but  there  are  cases  in  which 
they  will  compromise  the  AWE's  basic  objectives.  Understanding  the 
limitations  is  therefore  required  to  determine  if  the  NTC  is  the 
appropriate  site  for  the  AWE. 
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Five  Elements  of  NTC  Training 


NTC  training  comprises  five  basic  elements.  The  issues  associated  with 
each  element  will  be  discussed  in  this  section. 
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The  NTC  Training  Environment 

Desert  and  mountain  terrain  is  unpopulated 

Casualty  levels  are  unrealistic 

»  Battles  run  until  no  additional  training  opportunities  exist 
»  Fear  of  death  is  not  a  factor 

Safety  is  a  primary  concern 

»  There  are  few  force-on-force  night  battles 
»  Live-fire  scenarios  are  limited 

MILES  has  limitations 

»  Doesn't  fully  replicate  all  weapon  systems 
»  Affects  OPFOR  and  BLUEFORCE  differentially 


The  NTC  is  a  desert  warfare  environment  with  few  buildings  and  no 
indigenous  population.  Thus  it  is  an  inappropriate  site  for  exercises 
centering  on  urban  warfare,  jungle  terrain,  etc. 

As  a  training  facility,  the  NTC  approximates,  but  does  not  replicate, 
actual  combat.  In  particular,  two  characteristics  of  NTC  exercises  have 
important  implications  for  FRs  and  AWEs.  First,  battles  are  run  until 
no  further  training  opportunities  exist.  The  OPFOR  does  not  allow 
BLUEFORCE  units  to  retreat,  and  battles  are  often  fought  until  fewer 
than  10  percent  of  the  BLUEFORCE  units  are  alive.  This  means 
measures  of  effectiveness  such  as  force-exchange  ratios  are  suspect 
because  they  include  battle  segments  that  no  longer  represent  realistic 
scenarios.  Second,  in  NTC's  force-on-force  training  exercises,  the  units 
have  no  fear  of  dying.  Soldiers  frequently  perform  "heroic"  feats  that 
they  would  not  attempt  in  a  real  war.  This  does  train  the  troops  to  seize 
the  initiative  in  battle,  but  does  not  generate  realistic  data  for  the 
researcher. 

Safety  is  emphasized  at  training  sites.  Therefore,  there  are  few  force- 
on-force  night  battles  for  the  heavy  task  forces,  and  live-fire  scenarios 
are  limited. 

In  force-on-force  battles,  the  Multiple  Integrated  Laser  Engagement 
System  (MILES)  is  used  for  scoring,  and  the  relative  ability  of  OPFOR 
and  BLUEFORCE  to  exploit  MILES  has  a  substantial  effect  on  battle 
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outcomes.  For  example,  the  MILES  unit  that  replicates  the  TOW 
missile  system  is  difficult  to  align,  and  the  alignment  is  difficult  to 
maintain  over  time.  Through  constant,  long-term  practice,  the  OPFOR 
has  learned  how  to  properly  use  (TOW)  MILES.  The  BLUEFORCE 
does  not  have  this  capability.  Therefore,  TOW  exercises  performed  at 
the  NTC  are  compromised  by  MILES,  which  presents  a  fundamental 
problem  for  analysts. 

Aviation  assets  are  not  fully  instrumented  with  MILES.  Close  Air 
Support  currently  does  not  use  MILES  but  instead  is  scored  by 
observer-controllers.  Most  helicopters  have  some  form  of  MILES,  but  it 
does  not  replicate  their  weapon-system  capabilities  to  the  level  that 
other  MILES-equipped  weapon  systems  achieve.  Moreover,  MILES 
units  on  helicopters  often  fail  to  perform  to  specifications.  Overall,  this 
suggests  that  there  will  be  problems  in  FRs/AWEs  that  utilize  extensive 
aviation  assets. 
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OPFOR  Is  the  Best-Trained  Enemy  in  the  World 


»  Is  a  Soviet-style  enemy  with  limited  infantry 
strength  and  BMP  troop-carrying  capability 

»  Wins  five  out  of  six  engagements 

-  Uses  1960s-  and  1970s-era  equipment 

-  Training,  tactics,  experience,  and  procedures 
compensate  for  equipment  limitations 

-  MILES  can  make  OPFOR's  equipment  superior 
to  BLUEFORCE's  equipment 


An  FR  or  AWE  selected  for  the  NTC  must  take  into  account  the  unique 
nature  of  the  OPFOR.  It  is  the  best-trained  enemy  in  the  world.  It  is 
also  a  very  specific  type  of  enemy  whose  operations  are  based  on  Soviet 
doctrine.  A  pure  mechanized  force  with  limited  BMP  troop-carrying 
capability,  the  OPFOR  sometimes  uses  National  Guard  troops  or  other 
visiting  units  as  foot  soldiers,  primarily  deployed  with  AT-5s  to  defend 
a  pass  or  other  key  terrain. 

The  OPFOR  achieves  victory  in  five  out  of  six  battles.  It  wins  because  it 
trains  more  than  100  days  a  year  in  the  field,  has  tremendous 
knowledge  of  the  terrain  it  is  fighting  on,  and  knows  exactly  what  it 
and  the  BLUEFORCE  can  do  in  this  terrain.  Its  advantages  in  operating 
the  MILES  system  have  already  been  discussed. 

One  possible  approach  to  minimizing  the  OPFOR's  impact  on  the  AWE 
would  be  to  replace  it  with  a  BLUEFORCE  unit  for  that  rotation.  The 
unit  would  receive  the  same  amount  of  training  as  the  AWE  unit  but 
would  not  have  its  changes  in  DTOLMS.  The  AWE  would  then  more 
accurately  show  the  change  in  performance  possible  with  the  new 
DTOLMS. 
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Although  it  is  generally  understood  that  enhanced  training  is  provided 
and  advanced  weapons  are  acquired  to  help  assure  victory,  an 
underlying  premise  of  NTC  exercises  is  that  BLUEFORCE  must  do 
everything  right  to  win  against  the  OPFOR.  This  premise  enables 
highly  effective  training,  since  units  will  learn  better  and  more  lasting 
lessons  from  losing  than  from  winning.  As  a  result,  the  training  gained 
in  an  FR  or  AWE  conducted  at  the  NTC  is  not  likely  to  result  in  a 
BLUEFORCE  victory,  and  planners  must  take  this  into  account. 
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BLUEFORCE  Units  at  the  NTC  Are  Organized 
Differently  Than  for  Wartime  Deployment 


XX 


Division  assets 


AviatiojJ 
AxtMkrv 
CSS  ' 


i  I 


DTOC 
staffed 
bv  NTC 


Joint  Operations 
Air  Force  CAS 
Notional  Navy 
National  assets 
AF  &  Navy  recce 


i  1 


II 


□ 


Deploys  lo  NT'C 

Stays  at  home 
station 


Brigade  operations  are  a  fairly  recent  activity  at  the  NTC,  and  among 
the  echelons  participating  in  exercises  at  the  NTC,  brigades  are  the 
most  constrained.  They  deploy  to  the  NTC  with  more  division  and 
fewer  battalion  assets  than  they  might  normally  have  during  a  war. 
They  also  deploy  without  adjacent  or  supporting  brigades.  Their 
division  TOC  is  staffed  by  NTC  personnel,  and  it  generally  issues 
flawless  orders  and  only  slightly  distorted  versions  of  near-perfect 
intelligence. 

This  results  in  a  brigade  with  assets  it  does  not  normally  use.  Its 
operation  is  only  notionally  coordinated  with  that  of  adjacent  brigades; 
it  is  artificially  coordinated  with  the  division,  and  it  is  not  coordinated 
with  a  native  population  or  with  joint  forces.  The  Army's  experiences 
in  Panama,  Grenada,  Kuwait,  and  Somalia  indicate  that  coordination  is 
now  one  of  the  most  critical  tasks  to  ensure  brigade-level  success  in 
today's  conflicts. 

Maneuver  battalions  deploy  to  the  NTC  with  the  assets  they  would 
take  to  war.  Given  that  most  battles  are  at  the  battalion  level,  the  NTC 
battalion  receives  more  attention  from  the  brigade  than  it  might 
normally  receive  during  a  real  war.  There  are  also  times  when  the 
brigade  is  involved  with  the  other  battalion's  operations  and  gives  less 
attention  to  the  battalion.  Also,  for  most  battles  a  battalion  does  not 
need  to  coordinate  with  adjacent  battalions. 
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A  limited  number  of  joint  operations  are  performed  at  the  NTC.  Nellis 
Air  Force  Base  provides  A-lOs  and  F-16s  for  close  air  support.  Naval 
gunfire  is  notionally  provided  by  having  the  master  computer  in  the 
"Star  Wars"  building  fire  the  missions  and  score  their  effectiveness. 

Reconnaissance  assets  are  routinely  used.  Because  the  master  computer 
knows  all  vehicle  locations,  reconnaissance  assets  can  also  be  notionally 
deployed.  Upon  request,  the  division  can  receive  information  about 
what  a  recce  flight  would  have  seen  had  it  flown  a  given  mission. 

The  bottom  line  is  that  the  NTC  provides  realistic  battalion-level 
training,  but  training  at  the  higher  echelons  is  less  realistic.  FRs  and 
AWEs  should  account  for  the  fidelity  of  the  training  at  each  echelon. 
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There  Are  Four  Primary  Training 
Missions  at  the  NTC 


Armor  TF 


Mech  TF 


5  days 

6  days 

Live  fire 

Force-on-force 

6  days 

5  days 

Force-on-force 

Live  fire 

X 


3  days 


Brigade  FoF 


Missions  performed 

Defense-in-sector  Deliberate  attack 

Movement-to-contact  Hasty  attack 


BLUEFORCE  units  train  for  14  days  at  the  NTC.  At  the  start  of 
training,  one  battalion  goes  to  live-fire  exercises  while  the  other  goes  to 
force-on-force  exercises.  Each  task  force  (TF)  fights  three  battles.  Then 
they  switch.  After  11  days  the  TFs  unite  to  fight  two  force-on-force 
battles. 

Typically,  the  units  are  given  only  four  types  of  missions;  defense-in- 
sector,  movement-to-contact,  deliberate  attack,  and  hasty  attack. 
Although  special  units  such  as  division  cavalry  will  be  given  different 
missions — e.g.,  guard  or  screen  missions — appropriate  to  their  METE 
(mission  essential  task  list),  the  majority  of  battalions  perform  the  four 
missions  cited  above  during  their  NTC  rotation. 
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To  maximize  training  for  the  mounted  warfare  community, 
BLUEFORCE  missions  emphasize  the  direct  fire  fight.  Furthermore, 
the  fast-paced  OPFOR  is  a  difficult  enemy  on  which  to  call  indirect  fire. 
The  result  is  that  the  various  battlefield  operating  systems  (BOS)  are  not 
replicated  to  the  same  level.  This  limits  the  level  of  testing  for  AWEs/ 
FRs  that  seek  to  enhance  the  effectiveness  of  a  BOS  that  receives  less 
emphasis. 

An  example  is  Paladin,  the  latest  self-powered  artillery  deployed  by  the 
Army.  Field  artillery  is  of  limited  effectiveness  against  the  OPFOR 
because  of  a  lack  of  timely  target  reporting.  (The  OPFOR  tends  to  kill 
most  BLUEFORCE  forward  observers,  such  as  the  FIST  teams.) 

Because  its  capabilities  would  not  be  fully  demonstrated  at  the  NTC, 
Paladin  would  best  be  tested  elsewhere.  It  should  be  noted  that  if 
changes  were  made  to  more  fully  replicate  fire  support  at  NTC,  then 
equipment  like  Paladin  could  be  effectively  tested  at  NTC. 
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The  NTC  Data  Base  Supports  Training, 
Not  Analysis 


NTC  Data  Sources 


Digital  data 

Nondigital  data 

OC  observations 

•  Battle  replay  tapes 

•  Killer-victim 
scoreboard 

•  Combat  radio  nets 

•  Operation  orders  and 
written  fragmentary 
orders  (FRAGOs) 

•  Fire  logs 

•  After-action  review 
videos 

•  Take-home  packages 

•  Focused  OC  surveys 

The  data  base  was  developed  to  support  after-action  reviews 
The  data  base  is  difficult  to  use 

Observer-controller  records  are  a  valuable  source  of  data 


NTC  instrumentation  was  developed  to  give  OCs  the  necessary 
information  to  critique  their  counterparts.  The  positions  and  locations 
of  vehicles  during  the  battle  and  the  times  the  vehicles  fired  are 
recorded  in  battle  replay  tapes.  Roughly  20  percent  of  the  killer-victim 
pairings  are  found  by  the  "Star  Wars"  computers  and  are  displayed  on 
the  tape.  The  OCs  also  manually  enter  into  the  computer  the  end-of- 
battle  killer-victim  scoreboard.  Approximately  20  of  the  TF  combat 
radio  nets  are  recorded.  Written  operation  orders  and  FRAGOs  are 
archived.  Fire  logs  are  saved  by  the  artillery  OCs. 

These  data  are  extensively  used  for  after-action  reviews  and  take-home 
packages.  OCs  have  found  that  although  the  instrumentation  can  show 
what  happened,  their  direct  observations  of  the  battles  are  needed  to 
determine  why  those  events  took  place. 

RAND  researchers  have  found  the  NTC  data  sources,  in  general,  to  be 
incomplete  and  difficult  to  use.  To  adequately  quantify  a  battlefield 
issue,  RAND  has  utilized  focused  OC  surveys,  which  ask  from  10  to  25 
questions  about  specific  aspects  of  the  training  exercises  observed  by 
the  OCs.  Data  from  these  surveys  are  used  to  quantify  issues  and  to 
validate  or  reject  RAND  study  hypotheses.  RAND  has  found  that 
roughly  one  year's  worth  of  surveys  is  required  to  adequately  quantify 
an  issue. 
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NTC  is  currently  planning  to  upgrade  MILES  to  the  newest  version, 

S AWE-MILES  II  (surface  area  weapons  effect  MILES).  This  will  fully 
digitize  the  position-location  and  shooter-victim  data  set,  thereby 
allowing  additional  empirical  battlefield  data  to  be  available  for 
simulations. 

This  complete  data  set  may  also  permit  more  accurate  assessment  of  the 
performance  of  BLUEFORCE  units.  With  the  MILES  upgrade, 
measures  of  effectiveness  such  as  massing  of  firepower  and  fraction  of 
vehicles  engaging  the  enemy  will  be  easier  to  determine.  More 
importantly,  the  data  can  be  taken  as  a  function  of  time,  which  will 
solve  the  problem  of  mixing  data  taken  during  the  late,  "training-only" 
stages  of  the  battle  with  that  from  the  more  realistic,  early  part  of  the 
battle. 

Lastly,  the  digital  data  base  can  be  linked  to  Distributed  Interactive 
Simulation  (DIS)  systems,  which  could  potentially  result  in  more 
realistic  DIS  exercises.  Real-time  linkage  of  DIS  to  NTC  exercises  could 
also  expand  the  NTC's  training  environment  to  include  more  Army 
assets  and  perhaps  even  joint  operations. 
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Briefing  Outline 


»  Background 

»  Methodology  for  conducting  effective  AWEs  at 
the  NTC 

»  The  NTC  as  an  environment  for  experiments 

»  Strengthening  the  relationship  between  the  NTC 
and  the  analytic  community 


This  section  discusses  how  the  analytic  community  and  the  CTC  can 
each  benefit  from  a  stronger  relationship.  The  section  concludes  with  a 
summary  of  this  briefing. 
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The  Analytic  Community  Can  Help 
Make  FRs/AWEs  More  Effective 


Simulations  can 

»  identify  critical  features  that  can  and  ought  to  be 
measured 

-  Refine  objectives  into  MOEs 

-  Estimate  MOE  values  for  specific  FRs/AWEs 

»  compensate  for  unexpected  problems  with 
equipment  or  soldier  performance 

»  further  assess  the  qualitative  results  obtained  from 
FRs/AWEs 


The  Ml  A2  feasibility  example  showed  how  simulations  can  help 
determine  the  objectives  of  the  FR/ AWE,  and  they  can  be  used  to  refine 
those  objectives  into  MOEs  as  well  as  to  estimate  what  the  values  of  the 
MOEs  could  be  for  the  training  exercise. 

Simulations  can  also  be  used  to  demonstrate  "what  if's."  Murphy's 
Law  seems  to  apply  to  warfighting  exercises.  That  fact,  combined  with 
the  inherent  limitations  of  NTC,  suggests  that  it  is  unlikely  that  all  the 
objectives  of  the  FR  or  AWE  will  be  met.  Simulations  can  play  the 
battle  so  that  equipment  doesn't  break  and  troops  don't  make  training 
errors. 

Thus,  simulation  results  can  be  combined  with  "dirty  laboratory" 
results  from  NTC  to  form  a  more  complete  picture  of  how  the  FR/ AWE 
DTOLMS  changes  can  affect  BLUEFORCE  performance.  Simulations 
can  also  be  valuable  when  NTC  results  are  good  because  simulations 
can  extend  those  results  to  other  mission  scenarios. 
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The  Intensity  of  Training  at  the  NTC  Is  a 
Unique  Asset  to  the  Analytic  Community 

Provides  data  on  soldier  performance  in  a  highly  stressed 
environment 

»  Empirical  inputs  to  help  create  more  realistic 
simulations 

»  End-user  feedback  to  make  equipment  more  effective 
»  Investigate  new  concepts 


NTC  training  exercises  stress  the  soldiers  at  a  level  comparable  to  that 
of  war.  The  exercises  therefore  represent  a  rich  empirical  data  base  for 
simulations.  For  example,  in  measuring  how  long  it  takes  the  S2  to 
analyze  a  spot  report  and  give  the  TF  commander  an  update,  RAND 
researchers  found  that  a  well-trained  S2  takes  6  minutes,  while  others 
can  take  up  to  30  minutes.  Simulations  such  as  JANUS  could  use  these 
kinds  of  data  to  more  realistically  model  the  battlefield. 

The  high-stress  NTC  environment  can  also  provide  end-user  feedback 
on  new  equipment.  For  example,  soldiers  at  NTC  consistently 
indicated  that  the  viewing  reticule  of  the  Bradley  FLIR  was  too  bright. 
These  same  soldiers  had  not  identified  this  problem  during  home 
station  exercises.  The  only  other  time  the  problem  was  reported  was 
during  Desert  Storm. 

Even  without  FRs  and  AWEs,  NTC  can  help  qualitatively  assess  new 
concepts.  This  can  be  done  by  using  the  NTC  data  base;  seeking  OCs' 
comments  on  new  concepts;  or,  as  RAND  has  done,  having  the  OCs  fill 
out  a  short,  focused  survey  card  after  each  battle. 
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